Papers with activation patching
Spectra: A Mechanistic Interpretability Library for Vision-Language Models (2026.acl-demo)
Copied to clipboard
| Challenge: | Existing interpretability tools for visionlanguage models are limited to activation probing and saving. |
| Approach: | They propose a library specifically designed for mechanistic interpretability of visionlanguage models that provides unified abstractions for activation patching, attention pattern analysis, and meta-functions across diverse VLM architectures. |
| Outcome: | The proposed library handles architecture-specific complexities while maintaining a simple, high-level interface. |
From Reasoning to Answer: Empirical, Attention-Based and Mechanistic Insights into Distilled DeepSeek R1 Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Reasoning Models generate explicit reasoning traces alongside final answers . the extent to which these traces influence answer generation remains unclear . |
| Approach: | They conduct empirical evaluation of Large Reasoning Models that include explicit reasoning . they also show that answer tokens attend substantially to reasoning tokens . |
| Outcome: | The results show that including explicit reasoning improves answer quality across domains . they also show that answer tokens attend substantially to reasoning tokens - the authors . |
Can you map it to English? The Role of Cross-Lingual Alignment in the Multilingual Performance of LLMs (2026.eacl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) can answer prompts in many languages despite being pre-trained mostly on English text. |
| Approach: | They propose a Discriminative Alignment Index to quantify instance-level alignment across 24 languages other than English and three distinct NLU tasks. |
| Outcome: | The proposed model can perform natural language understanding tasks in 24 languages other than English and three distinct NLU tasks. |
Dissecting Fine-Tuning Unlearning in Large Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for fine-tuning-based unlearning are ineffective at completely erasing model-embedded knowledge, but their true effectiveness remains unclear. |
| Approach: | They propose to use activation patching and parameter restoration experiments to examine the limitations of fine-tuning-based unlearning methods for erasing harmful, sensitive, or copyrighted information within large language models. |
| Outcome: | The proposed methods alter the model’s knowledge retrieval process rather than genuinely erasing the problematic knowledge embedded in the model parameters. |
Are Knowledge and Reference in Multilingual Language Models Cross-Lingually Consistent? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Cross-lingual consistency should be considered to assess cross-lingual transferability, maintain factuality of model knowledge across languages, and preserve parity of language model performance. |
| Approach: | They examine pretrained and tuned models with code-mixed coreferential statements that convey identical knowledge across languages. |
| Outcome: | The proposed model shows different levels of consistency in multilingual models, subject to language families, linguistic factors, scripts, and bottlenecks on a particular layer. |
Towards Faithful Natural Language Explanations: A Study Using Activation Patching in Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are capable of generating persuasive Natural Language Explanations (NLEs) however, the faithfulness of these explanations should not be readily trusted at face value. |
| Approach: | They propose to use a causal mediation technique called activation patching to measure the faithfulness of an explanation towards supporting the explained answer. |
| Outcome: | The proposed metric, Causal Faithfulness, quantifies the consistency of causal attributions between explanations and the corresponding model outputs as the indicator of faithfulness. |
LLMs to Replace Crowdsourcing For Parallel Data Creation? The Case of Text Detoxification (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Manual annotation methods, such as crowdsourcing, are costly and require intricate task design skills. |
| Approach: | They propose to use open source LLMs to annotate parallel data for text detoxification . they generate a pseudo-parallel detoxification dataset using activation patching . |
| Outcome: | The proposed model performs comparable to the original dataset in automatic detoxification evaluation metrics and superior quality in manual evaluation and side-by-side comparisons. |
Information Flow Routes: Automatically Interpreting Language Models at Scale (2024.emnlp-main)
Copied to clipboard
| Challenge: | Current state-of-the-art language models (LMs) are built on top of the Transformer architecture. |
| Approach: | They propose to build graphs where nodes correspond to token representations and edges to computations . they show that attention heads and subword merging heads are important . |
| Outcome: | The proposed model can analyze behavior for specific types of predictions, or different domains. |
Dissecting Persona-Driven Reasoning in Language Models via Activation Patching (2025.findings-emnlp)
Copied to clipboard
| Challenge: | Large language models (LLMs) exhibit remarkable versatility in adopting diverse personas. |
| Approach: | They examine how assigning a persona influences a model’s reasoning on an objective task by activation patching . |
| Outcome: | The early Multi-Layer Perceptron (MLP) layers attend to syntactic structure of input and process its semantic content. |
Understanding and Mitigating Political Stance Cross-topic Generalization in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Recent studies have focused on the internal representations of large language models and the mechanisms that lead to unintended cross-topic generalization. |
| Approach: | They propose a method that uses inhibition to localize political neurons and a technique that uses topic-specific blocking to mitigate the cross-topic generalization. |
| Outcome: | The proposed method reduces cross-topic generalization by 20% while preserving topic-specific performance. |
Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformers (2025.acl-long)
Copied to clipboard
| Challenge: | A central question in multilingual language modeling is whether large language models develop a universal concept representation, disentangled from specific languages. |
| Approach: | They analyze latent representations during a word-translation task in transformer-based LLMs and extract the residual stream of the last token of the word to be translated and insert the mean at the corresponding positions in the forward pass. |
| Outcome: | The proposed model can translate a word in multiple languages without changing the language and vice versa. |
Discursive Circuits: How Do Language Models Understand Discourse Relations? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Discourse structure is essential for ensuring language models behave safely and ethically. |
| Approach: | They propose a task where a model completes a discourse given a specified relation . they propose CUDR task that enables activation patching to make circuit discovery feasible . |
| Outcome: | The proposed model recovers discourse understanding in the English PDTB-based CuDR task. |
Conceptual Hierarchies within LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing literature has explored abstraction within large language models (LLMs). |
| Approach: | They generate a dataset of semantic hierarchies and investigate their storage locations in six LLMs using activation patching, a causal intervention technique. |
| Outcome: | The results show that concepts at finer levels of granularity are stored around 61-78% of the time before those at coarser levels. |
Fine-Grained Analysis of Shared Syntactic Mechanisms in Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Recent advances in language models have demonstrated their ability to process linguistic expressions with complex syntactic structures. |
| Approach: | They investigate whether language models employ shared neural mechanisms across different constructions by applying causal interpretability methods at a granular level. |
| Outcome: | The proposed model performance improves on acceptability judgment benchmarks. |
Cell-Based Representation of Relational Binding in Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Recent work has found evidence that Large Language Models (LLMs) are able to track entities across discourse . however, the mechanism by which they bind entities, relations, and attributes remains unclear . |
| Approach: | They propose a low-dimensional cell-based binding representation for relational binding . they also show that context-specific CBR representations are related by translation vectors . |
| Outcome: | The proposed model encodes a low-dimensional cell-based binding representation (CBR) a translation vector in activation space enables cross-context transfer, the study shows . |